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DIGITAL IMAGE PROCESSING METHOD AND 



COMPUTER PROGRAM PRODUCT FOR 
DETECTING HUMAN IRISES IN AN IMAGE 



5 FIELD OF THE INVENTION 

The present invention relates to digital image processing methods 
for detecting facial features and more particularly to methods of detecting human 
irises in a digital image. 

BACKGROUND OF THE INVENTION 

10 In digital image processing it is often usefiil to detect human irises 

in an image. This information is used, for example, to locate other features in the 
image such as human eyes. This information can also be used for other purposes 
such as finding the orientation of a hiunan face in the image. 

Methods are known in the art for detecting human eyes. For 

15 example, U.S. Patent 6,072,892 discloses the use of a thresholding method to 
detect the position of human eyes in a digital image. In this method, a scanning 
window scans across the entire image using a raster scanning method. A 
histogram extractor extracts intensity histograms from the window as it scans 
across the image. Each intensity histogram is examined by a peak detector to find 

20 three peaks in the histogram representing the skin, the white of the eye, and the 
black of the pupil located within the iris. A histogram having the three peaks 
identifies a location in an image that potentially defines an eye position. Eye 
position is determined from among the potential locations by calculating the area 
under the histogram associated with each potential location and by selecting the 

25 location associated with the histogram having the largest area. 

One of the problems with this approach is that the entire image 
must be scanned on a pixel-by-pixel basis. Thus, a search window must be 
positioned at each pixel and a histogram must be assembled at each pixel location. 
Further, the area imder each histogram must be calculated and stored. It will be 

30 appreciated that this method consumes enormous amounts of computing power 
and reduces the rate at which images can be processed. This method can also 
produce a high rate of false positives. 
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Methods are also known to detect eyes that have abnormally high 
red content. Such abnormal red content is commonly associated with a 
photographic phenomenon known as red eye. Red eye is typically caused by a 
flash of light that is reflected by a pupil. As is described in commonly assigned 
5 and co-pending U.S. Patent application No. 08/919,560, it is known to search for 
images for pixels having the high red content that is indicative of red eye. 
Similarly, commonly assigned U.S. Patent 5,432,863 describes a user interactive 
method for detecting pixels in an image that have color characteristic of red eye. 
It will be recognized that these methods detect eyes only where red eye is present. 

1 0 Thus, there is a need for a method of locating human irises in a 

digital image that is accurate and efficient. 

SUMMARY OF THE INVENTION 
The need is met according to the present invention by a digital 
image processing method for detecting human irises in a digital image. This 

15 method comprises the steps of: measuring the red intensity of the pixels in the 
image, determining the probability that each pixel is an iris based upon the red 
intensity of the pixel, determining the probability that each pixel is not an iris 
based upon the red intensity of the pixel; and determining whether the pixel is an 
iris by analyzing the relationship between the probability that the pixel is an iris 

20 and the probability that the pixel is not an iris. In one embodiment of the present 
invention, the determination as to whether a pixel is an iris pixel is made based 
upon the application of a Bayes model to the probability that the pixel is an iris, to 
the probability that the pixel is not an iris, the probability of the occurrence of an 
iris in the image and probabiUty of the occurrence of a non-iris pixel in the image. 

25 In another embodiment of the present invention, skin color regions 

are detected in the image and only the pixels within the skin color regions are 
measured for red intensity. The determination as to whether a pixel is an iris pixel 
is made based upon the application of a Bayes model to the probability that the 
pixel is an iris, the probability that the pixel is not an iris, the probability of the 

30 occurrence of an iris in the skin color region and the probability of a non-iris in 
the skin color region. 
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The need is also met according to the present invention by a 
method for detecting irises in a digital image comprising the steps of finding an 
oval shaped skin color region, detecting iris color pixels in the oval shaped region 
using a Bayes iris model and locating eye positions based upon the detected iris 
5 color pixels. 

The need is also met according to the present invention by a 
computer program product for detecting irises in a digital image comprising: a 
computer readable storage medium having a computer program stored thereon for 
performing the method of the present invention. 
10 The present invention is effective to obtain iris positions in a 

frontal face image and has the advantage of reducing the region of the image that 
must be searched, thereby greatly reducing the computation required to locate an 
eye, and reducing the incidence of eye detection false positives. 

BRIEF DESCRIPTION OF THE DRAWINGS 
15 Figure 1 is a schematic diagram of an image processing system 

usefiil in practicing the present invention; 

Figure 2 is a flowchart illustrating the eye detection method of the 
present invention; 

Figure 3 is an illustration showing the relationship between certain 
20 geometric parameters and an oval shaped skin colored region in an image; 

Figure 4 is an illustration showing, the conditional probability that 
a given pixel is an iris pixel stated as a function of a specific red intensity and the 
conditional probabiUty that a given pixel is a non-iris pixel as a fimction of a 
specific red intensity I. 
25 Figure 4a is a flowchart presenting the process of developing a 

statistical model representing the conditional probabiHty that a given pixel is an 
iris pixel as a function of a specific red intensity level and the a statistical model 
representing the conditional probability that a given pixel is an iris pixel as a 
function of a specific red intensity level. 
30 Figure 5 is a flowchart illustrating the process of eye position 

detection after detecting the iris color pixel clusters; 

Figure 6 is an illustration showing the iris color pixel clusters; 



Figure 7 is a flowchart illustrating the details of the process of 
applying the method of summation of squared difference to eye position detection; 
and 

Figure 8 shows an eye template and a search window centered at 
5 the center of a cluster. 

DETAILED DESCRIPTION OF THE INVENTION 
Fig. 1, shows an image processing system useful in practicing the 
present invention including a color digital image source 10, such as a film 
scanner, digital camera, or digital image storage device such as a compact disk 
10 drive with a Picture CD. The digital image from the digital image source 10 is 

provided to an image processor 12, such as a programmable personal computer, or 
digital image processing work station such as a Sim Sparc workstation. The 
image processor 12 may be connected to a CRT display 14, an operator interface 
such as a keyboard 16 and a mouse 18. Image processor 12 is also connected to 
15 computer readable storage medium 17. The image processor 12 transmits 

processed digital images to an output device 19. Output device 19 can comprise a 
hard copy printer, a long-term image storage device, a connection to another 
processor, or an image telecommunication device connected for example to the 
internet. 

20 In the following description, a preferred embodiment of the present 

invention will be described as a method. However, in another preferred 
embodiment, the present invention comprises a computer program product for 
detecting human eyes and irises in a digital image in accordance with the method 
described. In describing the present invention, it should be apparent that the 

25 computer program of the present invention can be utilized by any well-known 
computer system, such as the personal computer of the type shown in Fig. 1. 
However, many other types of computer systems can be used to execute the 
computer program of the present invention. Consequently, the computer system 
will not be discussed in further detail herein. 

30 It will be understood that the computer program product of the 

present invention may make use of image manipulation algorithms and processes 
that are well known. Accordingly, the present description will be directed in 



particular to those algorithms and processes forming part of, or cooperating more 
directly with, the method of the present invention. Thus, it will be understood that 
the computer program product embodiment of the present invention may embody 
algorithms and processes not specifically shown or described herein that are 
5 useful for implementation. Such algorithms and processes are conventional and 
within the ordinary skill in such arts. 

Other aspects of such algorithms and systems, and hardware and/or 
software for producing and otherwise processing the images involved or co- 
operating with the computer program product of the present invention, are not 

1 0 specifically shown or described herein and may be selected from such algorithms, 
systems, hardware, components and elements known in the art. 

The computer program for performing the method of the present 
invention may be stored in a computer readable storage medium. This medium 
may comprise, for example; magnetic storage media such as a magnetic disk (such 

15 as a hard drive or a floppy disk) or magnetic tape; optical storage media such as an 
optical disc, optical tape, or machine readable bar code; soUd state electronic 
storage devices such as random access memory (RAM), or read only memory 
(ROM); or any other physical device or medium employed to store a computer 
program. The computer program for performing the method of the present 

20 invention may also be stored on computer readable storage medium that is 

connected to the image processor by way of the internet or other communication 
medium. Those skilled in the art will readily recognize that the equivalent of such 
a computer program product may also be constructed in hardware. 

Turning now to Fig. 2, the method of the present invention will be 

25 described in greater detail. Fig. 2 is a flow chart illustrating one embodiment of 
the iris detection method of the present invention. In the embodiment shown in 
Fig. 2, iris detection is accomplished by first detecting skin colored regions in the 
image and then identifying iris pixels from the skin colored regions. 

The first step in skin color detection is color histogram equalization 

30 shown in Fig. 2 as step 22. Color Histogram Equalization step 22 receives images 
to be processed and ensures that the images are in a form that will permit skin 
color detection. This step is made necessary because human skin may take on any 



number of colors in an image because of lighting conditions, flash settings or 
other circumstances. This makes it difficult to automatically detect skin in such 
images. In Color Histogram Equalization step 22, a statistical analysis of each 
image is performed. If the statistical analysis suggests that the image may contain 
5 regions of skin that have had their appearance modified by lighting conditions, 
flash settings or other circumstances then such images are modified so that skin 
colored regions can be detected. 

After the Color Histogram EquaUzation step 22, the image is 
searched for skin color regions in Skin Color Detection step 24. While it is 

1 0 possible to detect skin in a digital image in a number of ways, a preferred method 
for detecting skin in a digital image is to separated skin color pixels fi-om other 
pixels in an image by defining a working color space that contains a range of 
possible skin colors collected fi-om a large population of images. A pixel is then 
identified as skin color pixel if the pixel has a color that is within the working 

15 color space. 

Skin Color Detection step 24 identifies a region of skin color pixels 
in the image. This region can be defined in any number of ways. In one 
embodiment, the skin color region is defined by generating a set of pixel locations 
identifying the pixels in the image having skin colors. In another embodiment, a 

20 modified image is generated that contains only skin color pixels. In yet another 
embodiment. Skin Color Detection step 24 defines boundaries that confine the 
skin color region in the image. It will be recognized that more than one skin color 
region can be identified in the image. 

Oval Region Extraction step 26 examines the skin color regions 

25 detected by the Skin Color Detection step 24 to locate skin color regions that may 
be indicative of a face. Because the human face has a roughly oval shape, the skin 
color regions are examined to locate an oval shaped skin color region. When an 
oval shaped skin color region is found, the Oval Region Extraction step 26 
measures the geometric properties of the oval shaped skin color region. The Oval 

30 Region Extraction step 26 uses these measurements to define parameters that 
describe the size of the face and the location of the face within the image. 



Fig. 3 is an illustration of the relationship between the geometric 
parameters used to define an oval shaped skin color region in the image. As is 
shown in Fig. 3, these parameters include Oval_top 300, Oval_bottom 302, 
Oval left 304, Oval_right 306, Oval_center_row 308, and Oval_center_column 

5 310. These parameters are can be used in subsequent processing of the image. It 
will be recognized that the method of the present invention can be practiced using 
skin color detection regions that have shapes that are other than oval and that other 
geometric parameters can be defined in association with such shapes. It will also 
be recognized that it is not necessary to detect an oval or other shaped area in the 

10 image. In such case, the skin color region is examined to detect iris color pixels . 
Further, in such a case, still other parameters describing the skin color region are 
defined for use in the eye detection process. 

After the oval region extraction has been performed, Iris Color 
Pixel Detection step 28 examines the pixels in the oval shaped skin color region to 

1 5 detect iris color pixels. In the method of the present invention. Iris Color Pixel 
Detection step 28 determines whether a pixel is an iris by measuring the red 
intensity of the pixel. Red intensity levels are measured because it has been 
observed that that a human iris has a low red intensity level as compared to human 
skin which has a relatively high red intensity level. However, the method of the 

20 present invention does not use a red level thresholding method to determine 
whether a pixel is to be classified as an iris or as a non-iris. 

Instead, the present invention classifies a pixel as an iris or a non- 
iris pixel on the basis of a probability analysis. This probability analysis apphes 
an iris statistical model and a non-iris statistical model. The iris statistical model 

25 defines the probability that a given pixel is an iris pixel based upon the red 

intensity level of the pixel. Similarly, the non-iris statistical model defines the 
probability that a given pixel is not an iris pixel based upon the red intensity level 
of the pixel. The relationship between these models is non-linear as is shown by 
way of example in Fig. 4 which is an illustration of the conditional probability 402 

30 that a given pixel is an iris pixel stated as a fiinction of a specific red intensity and 
the conditional probability 404 that a given pixel is a non-iris pixel as a fiinction 
of a specific red intensity I. 
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The probability analysis can take many forms. For example, the 
probabilities can be combined in various ways with a pixel being classified as an 
iris or not on the basis of the relationship between these probabilities. However, 
in a preferred embodiment, a mathematical construct known as a Bayes model is 
5 used to combine the probabilities to produce the conditional probability that a 
pixel having a given red intensity belongs to an iris. 

In this embodiment, the Bayes model is apphed as follows: 

. P{l I iris)p{iris) 

\}ris I )- p^j^ iris)p{iris) + P{l \ noniris)p{noniris) ' 

10 

where P(iris \ l) is the conditional probability that a given pixel intensity belongs 
to an iris; P{l \ iris) is the conditional probability that a given iris pixel has a 
specific intensity I; Pi^ris) is the probability of the occurrence of an iris in the 
face oval region; P(l \ noniris) is the conditional probability that a given non-iris 
1 5 pixel has a specific intensity I; and P{noniris) is the probability of the occurrence 
of a non-iris pixel in the face oval region. The Bayes model further apphes the 
probability of the occurrence of an his in a face oval region and the probability of 
the occurrence of a non-iris pixel in the face oval region. Using a probability 
analysis based on the Bayes model, a pixel is classified as an iris if the conditional 
20 probability that a pixel having a given red intensity belongs to an iris is greater 
than, for example, 0.05. 

In the embodiment described above, only those pixels in the oval 
shaped skin color region defined by Ovaljop 300, Oval_bottom 302, Ovaljeft 
304, and Oval_right 306 are examined. Confining the pixels to be examined to 
25 those in the oval shaped skin color region reduces number of pixels to be 
examined and decreases the likelihood that pixels that are not irises will be 
classified as such. It will be understood that confining the pixels to be examined 
to those in the skin color region, as is described in other embodiments of the 
present invention, conveys similar advantages. It will also be understood that it is 
30 not necessary to detect skm color regions and that the present mvention can be 



practiced by measuring the red intensity of each pixel in the image and 
determining whether a pixel is an iris based upon the probability analysis 
described above. 

Fig. 4a shows a flow chart illustrating the processes used in the Iris 
5 Color/Bayes Model Training step 27 of Fig. 2 for developing the statistical models 
used to classify the pixels. This step will be performed before the method for 
detecting irises is used to detect iris pixels. As is shown, a large sample of frontal 
face images are collected and examined. All iris pixels and non-iris pixels in the 
face region of each image are then manually identified 40, 42. Next, the 

10 conditional probability that a given iris pixel has a specific red intensity I, 

P{l I iris) is computed and the probability of the occurrence of an iris in the face 
oval region, P{iris) 44 is computed; then the conditional probability that a given 
noniris pixel has a specific red intensity I, P{l \ noniris) is computed and finally 
the probability of the occurrence of a non-iris pixel in the face oval 

15 region, P{noniris) 46 is computed. The computed statistical models of iris and 

non-iris ^e used in the Bayes formula to produce the conditional probability that a 
given pixel intensity belongs to an iris, P{iris | /) 48. In application, the Bayes 
model can be used to generate a look-up table to be used in Iris Color Pixel 
Detection step 28. 

20 The Iris Color Pixel Detection step 28 identifies the location of the 

iris color pixels in the image. In some cases, it will be desirable to ensure that the 
iris color pixels that are detected are associated with an eye. This is done by 
performing the step of eye detection. Eye Position Detection step 27 is used to 
estimate the eye positions. It will be appreciated that there are many ways to 

25 determine whether an iris pixel is associated with an eye in the image. In one 
preferred embodiment of the present invention, the iris color pixel locations are 
used to facilitate the process of determining whether an iris pixel is associated 
with an eye in the image. 

Fig. 5 shows a flow chart illustrating the process of eye position 

30 detection using the iris color pixels. As is shown in Fig. 5, the eye position 

detection process starts with an Iris Pixel Clustering step 50. If iris color pixels 
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are detected, then the iris pixels must be assigned to a cluster. A cluster is a non- 
empty set of iris color pixels with the property that any pixel within the cluster is 
also within a predefined distance to another pixel in the cluster. One example of a 
predefined distance is one thirtieth of the digital image height. The Iris Pixel 
Clustering step 50 of Fig. 5 groups iris color pixels into clusters based upon this 
defmition of a cluster. However, it will be understood that pixels may be 
clustered on the basis of other criteria. 

Under certain circumstances, a cluster of pixels may not be vahd. 
Accordingly, the optional step of vaUdating the clusters is shown in Fig. 5 as 
Cluster VaUdation step 51. A cluster may be invalid because, if it contains too 
many iris color pixels or because the geometric relationship of the pixels in the 
cluster suggests that the cluster is not indicative of an iris. For example, if the 
ratio is greater than two, then this cluster is invahd. Invalid iris pixel clusters are 
removed fi-om fiirther consideration by the method of the present invention. 
Accordingly, in the portions of the description that follow, vaUd iris pixel clusters 
will be referred to simply as iris pixel clusters. 

After the clustering operation, a center for each of the clusters is 
calculated in Cluster Centering step 52. The center of a cluster is determined as 
the center of mass of the cluster. The center position of the clusters is calculated 
with respect to the origm of the image coordinate system. The origin of the image 
coordinate system for a digital image is defined is at the upper left comer of the 
image boundary. 

Oval division step 54 uses the Oval_center_column 310 parameter 
to separate the oval shaped skin color region into a left-half region and a right-half 
region. As is shown in Fig. 6 iris pixel clusters 602 and the center positions 600 
of the iris pixel clusters 602 are positioned in either the left-half or right-half 
regions 604 and 606 separated by the Oval_Center_Column 310. 

To locate eyes in the image using the iris pixel clusters, a left-eye 
position search 56 is conducted in the left-half region using a method known as 
the Summation of the Squared Difference. A right-eye position search 58 is also 
conducted in the right-half region based on the same Summation of the Squared 
Difference method. In general, the summation of the squared difference method 
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involves calculating the summation of the squared difference of the intensity 
values of the corresponding pixels in an eye template and a patch of the image that 
has the same size as the template. In this method, each pixel in the patch of pixels 
has a corresponding pixel in the template. The difference between the intensity 
5 level of each of the corresponding pixels is calculated. Each difference is then 
squared. The sum of each of the squared differences for each of the pixels in the 
set is then calculated. This summation of the squared differences provides a 
relative measure of the degree of correspondence between each of the pixel sets 
measured and the template. The eye template itself is generated by averaging a 
1 0 large number of sample eye images. 

As appUed in the present invention, summation of the squared 
difference values are calculated for each pixel in each window in each half-region. 
These values are compared and the pixel having the lowest relative summation of 
the squared difference value is identified as an eye location for the half-region. 
1 5 This process is performed separately on the clusters of the left and the right-half 
regions of the image in the maimer described below. 

It will be noted that while the present invention has been described 
as using the simunation of the squared difference method to identify the best 
relative match between the average eye template and each of the pixels, other 
20 methods to determine the degree of relative correspondence can be used. In 

particular, the mean-squared error method can be used in place of the simimation 
of the squared difference method. 

Referring to Figs. 7 and 8, the eye position search process 56 or 58 
is started with centering 70 a window 800 at the center of each cluster 802 in a 
25 respective half-region. The operation of calculating the summation of the squared 
differences 72 is then performed, separately, using a patch of pixels centered on 
each of the pixels in each of windows 74. The position of the pixel having the 
lowest summation of squared difference value in each window 800 is recorded 76. 
When this process has been completed for every cluster center 802 in a half region 
30 78, the position of the pixel having the lowest summation of squared difference 
value is recorded 79. This position is the eye position for the half-region. 
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It will be appreciated that the summation of the squared difference 
method of steps 56 and 58 of Fig. 5, can also be performed without the use of oval 
region extraction. In such an embodiment, the skin colored region can be divided 
into a left-half region and a right-half region. Iris pixel clusters can then be 
5 divided into left-half region and right-half region clusters. The siunmation of the 
squared difference method can then be appUed. 

The subject matter of the present invention relates to digital image 
understanding technology, which is understood to mean technology that digitally 
processes a digital image to recognize and thereby assign usefiil meaning to 
10 human understandable objects, attributes or conditions and then to utilize the 
results obtained in the fiirther processing of the digital image. 

In this manner, the present invention provides an efficient method 
for detecting normally appearing human irises in a digital image. 

The invention has been described in detail with particular reference 
1 5 to certain preferred embodiments thereof, but it will be understood that variations 
and modifications can be effected within the spirit and scope of the invention. 
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PARTS LIST 



10 


digital image source 


12 


image processor 


14 


display 


16 


keyboard 


17 


computer readable storage medium 


18 


mouse 


19 


output device 


22 


color histogram equalization step 


24 


skin detection step 


26 


oval region extraction step 


27 


iris color Bayes model training step 


28 


iris color pixel detection step 


29 


eye position detection step 


40 


iris pixels 


42 


non-iris pixels 


44 


computing step 


46 


computing step 


48 


computing step 


50 


iris pixel clustering step 


51 


cluster validation step 


52 


cluster centering step 


54 


oval division step 


56 


left eye position search 


58 


right eye position search 


70 


window centering 


72 


summation of squared difference calculation 


74 


checking step 


76 


position recording step 


78 


checking step 


79 


position recording step 


300 


Ovaltop 
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302 Oval_bottom 

304 Ovaljeft 

306 Oval_right 

308 Oval_center_row 

310 Oval_center_column 

402 probability curve 

404 probability curve 

600 cluster center 

602 iris color pixel 

604 left half region 

606 right half region 

800 a window 

802 average eye template moving in window 

804 an image 

806 average eye template 



